System and method for determining spatial coordinates

ABSTRACT

The invention relates to a system for determining spatial coordinates of points, wherein the system involves a fixed array of a network ( 2 ) of control points ( 3 ), and a manually or mechanically manipulatable measuring probe ( 1; 1′ ) containing at least one camera ( 9′, 9″, 9′″ ), and wherein the measuring probe ( 1; 1′ ) is designed for physical contact with a measuring point on an object ( 4 ) which is to be measured via a fixedly mounted contact unit ( 14 ) projecting from the measuring probe, which has a known, defined position in the coordinate system of the measuring probe; and a system unit ( 5 ) which computes the position of the contact unit ( 14 ) relative to a coordinate system defined by said control points ( 3 ) on the basis of the measuring probe&#39;s ( 1; 1′ ) image of the control points ( 3 ), and the known position of the contact unit ( 14 ) in the coordinate system of the measuring probe ( 1; 1′ ). There are also methods for calibrating the measuring probe, determining the position of control points ( 3 ) in a network ( 2 ) and calibrating the contact unit ( 14 ) on the measuring probe ( 1; 1′ ).

This application is the national phase of international applicationPCT/NO97/00189 filed Jul. 21, 1997 which designated the U.S.

FIELD OF THE INVENTION

The present invention relates to a system for determining spatialcoordinates of points, and methods related to the calibration of a probeand its contact unit, and to the determination of the position of pointsin a network.

DESCRIPTION OF THE RELATED ART

Norwegian Patent No. 174 025 describes a system for determining spatialcoordinates of points on a face or an object, where the measuring probeis made in the form of a light pen having light sources at knowncoordinates relative to a probe-fixed coordinate system, and where thespatial coordinates are determined by registering the image of the lightsources on a sensor in a single stationary camera. The position of thelight pen is thus determined in a coordinate system given by theposition of the camera. The system results in limited accuracy, given bythe dimensions of the light pen and the visual field of the camera.

Norwegian Patents Nos. 164 946 and 165 046 describe systems and methodsbased on two or more stationary or movable cameras. Great accuracy isattained by using these in that the measuring point is observed from twoor more camera positions.

The object of the present invention is to redress the limitationsassociated with the known solutions. The system according to the presentinvention is thus characterised by the features which are set forth inthe patent claims below. The methods mentioned above are alsocharacterised by the features set forth in the patent claims below.These and additional distinctive features of the invention will also beset forth in the following description with reference to the appendeddrawings.

New, miniaturised cameras make possible a hand-held probe containing acamera, and where its position and orientation are determined on thebasis of registering an image of a fixedly mounted network of referencepoints. The difference from the previous system as described inNorwegian Patent No. 174 025 is thus that a network of reference pointsis stationary relative to the object which is to be measured, whilst theprobe, which according to the present invention contains one or morecameras, is moved across the object which is to be measured.Considerably higher accuracy is attained with this solution than withthe previously patented system based on one camera and light pen. Thisis due to the fact that images of several points can be made at eachregistration, and their geometrical distribution can be adapted suchthat they are distributed across the entire visual field of the cameraand at different distances from the camera-based probe.

The accuracy attainable with this system solution is comparable to whatwas previously attainable with a minimum of two cameras which view theobject which is to be measured from two different directions ofobservation. This invention thus provides a system which is far simpler,is low in weight and complexity, and low in production costs.

The probe can be made wireless in that the image is transmitted from thecamera to a central unit by radio signals or infrared communication.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1a and 1 b show the measuring system and the use thereof.

FIGS. 2a, 2 b and 2 c show a camera-based probe respectively in sideview, top view and from below.

FIG. 3 shows a simplified, ideal camera model.

FIG. 4 shows determination of camera position in a network having knowncoordinates.

FIG. 5 shows how the position of the contact unit can be determined.

FIGS. 6a and 6 b shows a probe with several camera units.

FIG. 7 illustrates the use of the invention for three-dimensionalmeasuring in connection with a three-dimensional object, e.g., a carmodel.

FIG. 8 shows the principle of wireless transmission of signals betweenprobe and system unit.

FIG. 9 shows the probe mounted on a robot.

The measuring system and the use thereof are outlined in FIGS. 1a and 1b. The system consists of a measuring probe 1 based on one or morecameras in one unit, and a network 2 of control points 3, a system unitcontaining computer unit 5 and an operator console 6, 7. Usually, theoperator console will consist of a keyboard 6 and a monitor 7. Themeasuring probe 1 is normally connected to the system unit 5 via a cable8. The principle of measurement is that the camera-based probe 1 touchesthe object 4 at the point which is to be coordinate-determined, that thecamera in this position makes images the whole of or parts of thenetwork 2 of control points 3, and that the spatial position of theprobe is computed on the basis of the registered image.

As shown in FIGS. 2a, 2 b and 2 c, the probe consists of a camera 9 withhandle 12 mounted thereon, exposure button 13 and contact unit 14.Normally, the camera is a video camera 10, e.g., based on CCD (ChargeCoupled Device) or CID (Charge Injected Device) sensor technology, withlens 11. The exposure button 13 is manipulated to actuate an imagemaking of the network 2 of control points 3. The contact unit 14 is areplaceable unit, e.g., in the form of a tip or a sphere. The positionof the probe 1 is related to the centre of the contact unit 14. Thispresupposes that the position of the centre is known relative to thecoordinate system of the probe 1. Method for this are described below.If the contact unit 14 is a sphere, the analysis of the measurementresults must include compensation for the radius of the sphere. Thiscould be avoided by instead using a tip-shaped contact unit.

According to the camera model which is outlined in FIG. 3, the optics 11of the camera 9 may be represented as a tiny hole, a so-called “pinhole”17. Images of all points 3 are recorded on the sensor 10 as an idealrectilinear projection through this hole 17 (the centre of projection)of an image point 18. In a practical camera, lens error such asdistortion will involve this straight line from the object point beingdeflected through the lens. It is possible to compensate for the errorwhich occurs as a result of this deviation from the simplest cameramodel by calibrating the camera. In this context calibrating meansdetermining the relation between the true image of a point on thesensor, and the spatial direction to the point relative to acamera-fixed coordinate system.

A network 2 of control points 3 is mounted around the object which is tobe measured. In FIGS. 1 and 2 control points are only given in one planeon one side of the object which is to be measured. It would beadvantageous to have control points on several sides of the object,optionally above and/or below, and at different distances from theobject. This would give improved accuracy and ensure unambiguousmathematical solutions. The control points are so formed that they areeasily identifiable in relation to the background. This may be as pointsor patterns (circles, crosses) painted in white on a black background,retroreflective measuring points, so-called targets, or active lightsources (e.g., light emitting diodes). Other easily identifiablepatterns may also be used. To register retroreflective targets thecamera or probe should have a powerful light source mounted thereon,e.g., a flash unit 15, which illuminates the points 3 when creatingimages. When using active light sources instead of retroreflectivetargets, these may be on continuously, or may be switched onsynchronously with the exposure control of the camera.

The network 2 may be mounted on a movable object, e.g., a frame which isbrought to the object that is to be measured.

The position of the individual control points 3 in the network 2 must beknown with the greatest accuracy possible in order to be able to computethe position of the probe 1 on the basis of a single image. Methods fordetermining the position of the control points 3 in relation to oneanother are described below.

The determination of camera position relative to the network 2 of pointshaving known coordinates will now be described in more detail. Tounderstand this description it is essential to distinguish between thefollowing coordinate systems:

The network's spatial coordinate system. The coordinates of the networkpoints 3 are known relative to this coordinate system.

The internal coordinate system of the camera (probe). All registrationsby the camera's sensor are related to this coordinate system.

The coordinate system of the object which is to be measured. Whenmeasuring, e.g., points on a car body, it is desirable for these to berelated to a car-fixed coordinate system. A coordinate system of thiskind is defined by specific points on the object, where these points aredefined as having certain coordinate values.

The image of the network 2 of control points 3 gives a number of imagepoints 18 on the sensor 10 of the camera as shown in FIG. 4. On thebasis of these registered image points in the coordinate system of thecamera (probe) and knowledge of the spatial coordinates of the controlpoints 3 in a stationary, space-fixed coordinate system, it is possibleto compute the position and orientation of the probe 1 relative to thestationary spaced-fixed coordinate system 2.

There are different methods for carrying out this computation. Thesimplest would be to make a direct coordinate transformation between thecoordinates of the image points in the camera-fixed coordinate systemand the associated known coordinates of the points in the network'scoordinate system. An accurate and approved method consists of making abundle adjustment as known from photogrammetric technique.

An efficient computation presupposes that it is known or rapidlydiscovered which image points correspond to which points in the network.This may be based on the points in the network being arranged in easilyidentifiable patterns, and that standard algorithms for patternrecognition are used for identification.

There are no limitations on the orientation of the probe 1, but theaccuracy of determination of its position and orientation will dependupon how many of the control points 3 are made images of, and thespatial distribution of these. The image making of network points musttake place so quickly that vibrations or the operator's movements do nothave any effect upon the accuracy of measurement. The exposure time willtypically be about {fraction (1/1000)} of a second.

A method of calibrating a video camera to determine spatial direction isdescribed in Norwegian Patent No. 165 046. Within the field ofphotogrammetry other methods are known that are based on a network ofpoints in known or unknown positions being photographed from a camera indifferent positions and orientations relative to the network. Thecalibration of the camera consists of a number of correction parametersbeing determined, so that the image of one point may be considered as anideal projection through the projection centre of the camera. As analternative to correction parameters, a calibration table could beestablished corresponding to the relation between the real and idealimage of a point.

Photogrammetric calibration methods are described in, e.g., H. M. Kamara(Ed.): Non-topographic photogrammetry. Second Edition, 1987.

Calibration of the contact unit relative to a probe-fixed coordinatesystem will now be described in more detail. When measuring a point, theposition of the probe 1 is required to be related to the centre of thecontact unit 14. This requires knowledge of the position of the contactunit 14 relative to the coordinate system of the probe 1 (camera). Asimple method of determining the position of the contact unit 14 isoutlined in FIG. 5. This consists of holding the contact unit 14 at afixed point, whilst the probe itself is moved around this fixed point. Anumber of registrations (e.g., as in the illustrated example) of theposition and orientation of the probe 1 (camera) relative to the network2 are made for different orientations of the probe. By combiningregistered position and orientation of the probe 1 for each registrationwith the information that the point of contact 14 was held still, theposition of the centre of the contact unit 14 can be determined relativeto both the network's coordinate system 2, and to the internalcoordinate system 1 of the probe.

This method is so simple and so fast to carry out that it makes possiblethe use of replaceable contact units in order to allow varied lengthsand orientations of the contact unit relative to the rest of the probe.

The coordinates of the individual points 3 in the network 2 can bedetermined by making images of the control points 3 from differentdirections by means of the probe unit 1. This requires the observationof each individual control point 3 from a minimum of two differentdirections. This procedure can be carried out as a separate process byusing a precalibrated probe unit, or can be combined with thecalibration of the probe.

To be able to give the network's dimensions a correct scale, thedistance between a minimum of two of the points in the network must beknown, so that it can used as a scale factor in the bundle adjustment.If there are no known distances between fixed points in the network, itis possible by measuring the network to provide a distance standard inthe network. The distance standard may be in the form of a rod 21 havingcontrol points of the same type as in the network. The distance betweenthese control points must be known.

The accuracy in the determination of the position and orientation of theprobe will be influenced by a number of geometrical factors:

dimension of the network, and distance between probe and network

density and form of control points in the network

the visual field of the camera (aperture angle)

Improved accuracy may be attained by mounting several camera units withassociated lenses 11′, 11″, 11′″ together in a single probe as shown inFIGS. 6a and 6 b. These may have wholly or partly overlapping visualfields as indicated in FIG. 6a, or different fields as shown in FIG. 6b.The whole probe will have the same basic mode of operation as describedabove for a probe containing a single camera. The observations from theindividual cameras will all be related to a common probe-fixedcoordinate system. The image making of the network must be donesynchronously by the individual cameras.

The calibration method described for a probe based on one camera can beextended to handle several cameras in one unit. These will haveindependent calibration parameters. In addition, position andorientation of the individual cameras can be determined relative to acommon probe-fixed coordinate system.

FIG. 7 shows how the probe 1′ can be used to measure points on thesurface of a car, and wherein the network 2 is divided into severalsub-networks 2′, 2″ and 2′″ of points 3.

To avoid problems with cabling to the probe, the probe can be madewireless by mounting therein a module 16, 17 to transmit the image to areceiver module 18, 19 on the central unit via radio signals or infraredcommunication. It is not necessary to feed control signals back to theprobe unit. FIG. 8 shows the probe 1 connected to a transmitter 16having antenna 17, and where the system unit 5 is equipped with areceiver 18 having antenna 19. Although the configuration is intendedprimarily for radio signals, it will be understood that the arrangementwith some modifications is just as suitable for infrared communication.

The measuring method can be automated by mounting the probe on a robot20 or another mechanical device to guide the probe to a number ofselected measuring points. The probe 1 may contain a sensor unit 21 forregistering that the contact unit 14 is in physical contact with theobject. When contact is registered, the position and orientation of theprobe are automatically registered by making an image of the network 2.

What is claimed is:
 1. A system for determining spatial coordinates ofpoints, wherein the system involves a fixed array of a network ofcontrol points, comprising: a manually or mechanically manipulatablemeasuring probe containing at least one camera and wherein the measuringprobe is designed for physical contact with a measuring point on anobject which is to be measured via a fixedly mounted contact unitprojecting from the measuring probe, which has a known, defined positionin the coordinate system of the measuring probe; and a system unit whichcomputes the position of contact unit relative to a coordinate systemdefined by said control points on the basis of the measuring probe'simage making of the control points, and the known position of thecontact unit in the coordinate system of the measuring probe, whereinthe measuring probe has a contact sensor which is functionally connectedto the contact unit for registering mechanical contact between thecontact unit and the object which is to be measured at the measuringpoint, the contact sensor on said registration causing automaticposition measurement.
 2. A system for determining spatial coordinates ofpoints, wherein the system involves a fixed array of a network ofcontrol points, comprising: a manually or mechanically manipulatablemeasuring probe containing at least one camera and wherein the measuringprobe is designed for physical contact with a measuring point on anobject which is to be measured via a fixedly mounted contact unitprojecting from the measuring probe, which has a known, defined positionin the coordinate system of the measuring probe; and a system unit whichcomputes the position of contact unit relative to a coordinate systemdefined by said control points on the basis of the measuring probe'simage making of the control points, and the known position of thecontact unit in the coordinate system of the measuring probe, whereinthe measuring probe is equipped with a powerful light source.
 3. Asystem as disclosed in claim 2, wherein the light source is a flashunit.
 4. A system for determining spatial coordinates of points, whereinthe system involves a fixed array of a network of control points,comprising: a manually or mechanically manipulatable measuring probecontaining at least one camera and wherein the measuring probe isdesigned for physical contact with a measuring point on an object whichis to be measured via a fixedly mounted contact unit projecting from themeasuring probe, which has a known, defined position in the coordinatesystem of the measuring probe; and a system unit which computes theposition of contact unit relative to a coordinate system defined by saidcontrol points on the basis of the measuring probe's image making of thecontrol points, and the known position of the contact unit in thecoordinate system of the measuring probe, wherein said control pointsconsist of one or more of the following types: active light sources,passive light sources, light reflecting targets, easily identifiableobjects or patterns, and said network of control points being mounted ona movable object.
 5. A method for calibrating a manually or mechanicallymanipulatable measuring probe in a system intended for determiningspatial coordinates of points, the system having a fixed array of anetwork of control points, at least one camera on said measuring probe,the measuring probe being designed for physical contact with a measuringpoint on an object which is to be measured via a fixedly mounted contactunit projecting from the measuring probe, which has a known, definedposition in the coordinate system of the measuring probe; and a systemunit for computing the position of the contact unit relative to acoordinate system defined by said control points on the basis of themeasuring probe's image making of the control points, and the knownposition of the contact unit in the coordinate system of the measuringprobe, the method comprising the steps of: moving the camera or camerasof the measuring probe to a number of different observation directionsor orientations; successively recording in the camera or cameras imagesof the fixed array of the control points in the network from saiddifferent observation directions or orientations; and correlating thesuccessive images with one another to establish a number of correctionparameters or a calibration table corresponding to the relation betweenthe real and ideal image of a point which is due to the opticalproperties of the camera or cameras.
 6. A method as disclosed in claim5, wherein the measuring probe has at least two cameras, furthercomprising the step of: causing the optical properties of each camera tobe determined by bundle adjustment.
 7. A method for determining theposition of control points in a fixed array of a network of controlpoints which is a part of a system used for determining spatialcoordinates of points, the system further having a manually ormechanically manipulatable measuring probe containing at least onecamera, the measuring probe being designed for physical contact with ameasuring point on an object which is to be measured via a fixedlymounted contact unit projecting from the measuring probe, which has aknown, defined position in the coordinate system of the measuring probe;and a system unit for computing the position of the contact unitrelative to a coordinate system defined by said control points on thebasis of the measuring probe's image making of the control points, andthe known position of the contact unit in the coordinate system of themeasuring probe, the distance between at least two of the control pointsbeing known, the method comprising the steps of: moving the camera orcameras of a calibrated measuring probe to at least two differentdirections of observation; recording in the camera or cameras an imageof the fixed array of the control points in the network from saiddifferent directions of observation; and establishing a table ofcoordinate values for the control points in a coordinate system for thenetwork defined by a selection of the control points.
 8. A method asdisclosed in claim 7, further comprising the step of: causing thespatial coordinates of the control points to be determined by bundleadjustment.
 9. A method for calibrating a manually or mechanicallymanipulatable measuring probe in a system used for determining spatialcoordinates of points, the system having a fixed array of a network ofcontrol points, at least one camera on said measuring probe, themeasuring probe being designed for physical contact with a measuringpoint on an object which is to be measured via a fixedly mounted contactunit projecting from the measuring probe, which has a known, definedposition in the coordinate system of the measuring probe; and a systemunit for computing the position of the contact unit relative to acoordinate system defined by said control points on the basis of themeasuring probe's image making of the control points, and the knownposition of the contact unit in the coordinate system of the measuringprobe, where the poition of control points in a network which form partof the system are determined concurrently, the distance between at leasttwo of the control points being known, the method comprising the stepsof: moving the camera or cameras of the calibrated measuring probe to atleast two different directions of observation; recording in the cameraor cameras an image of the fixed array of the control points in thenetwork from said different directions of observation; and establishinga table of coordinate values for the control points in a coordinatesystem for the network defined by a selection of the control points andin the same computation correlating the images for establishing a numberof correction parameters or a calibration table corresponding to therelation between the real and ideal image of a point.
 10. A method forcalibrating a contact unit on a manually or mechanically manipulatablemeasuring probe in a system used for determining spatial coordinates ofpoints, the system having a fixed array of a network of control points,at least one camera on said measuring probe, the measuring probe beingdesigned for physical contact with a measuring point on an object whichis to be measured via a fixedly mounted contact unit projecting from themeasuring probe, which has a known, defined position in the coordinatesystem of the measuring probe; and a system unit for computing theposition of the contact unit relative to a coordinate system defined bysaid control points on the basis of the measuring probe's image makingof the control points, and the known position of the contact unit in thecoordinate system of the measuring probe the method comprising the stepsof: moving the camera or cameras of the calibrated measuring probe to anumber of different observation directions or orientations, while itscontact unit remains in contact with the fixed reference point ormeasuring point on an object; registering the position and orientationof the measuring probe relative to a network of control points in thatin the camera or cameras there is recorded an image of the fixed arrayof the control points in the network for each of said directions ororientations; and combining registered position and orientation of themeasuring probe for each registration with information that the point ofcontact for the contact unit is the same, whereby the position of thecenter of contact unit is determined in relation to the network'scoordinate system and the internal coordinate system of the measuringprobe.
 11. A system as disclosed in claim 4, wherein the measuring probehas a contact sensor which is functionally connected to the contact unitfor registering mechanical contact between the contact unit and theobject which is to be measured at the measuring point, the contactsensor on said registration causing automatic position measurement. 12.A system as disclosed in claim 4, wherein the measuring probe isequipped with a powerful light source.
 13. A method for calibrating ameasuring probe in a system used for determining spatial coordinates ofpoints, the system including: a fixed array of a network of controlpoints, a manually or mechanically manipulatable measuring probecontaining at least one camera, the measuring probe being designed forphysical contact with a measuring point on an object which is to bemeasured via fixedly mounted contact unit projecting from the measuringprobe, which has a known, defined position in the coordinate system ofthe measuring probe; and a system unit for computing the position of thecontact unit relative to a coordinate system defined by said controlpoints on the basis of the measuring probe's image making of the controlpoints, and the known position of the contact unit in the coordinatesystem of the measuring probe, the method comprising: moving the cameraor cameras of the measuring probe to a number of different observationdirections or orientations; successively recording in the camera orcameras images of the fixed array of the control points in the networkfrom said different observation directions or orientations; andcorrelating the successive images with one another to establish a numberof correction parameters or a calibration table corresponding to therelation between the real and ideal image of a point which is due to theoptical properties of the camera or cameras.
 14. A method as disclosedin claim 13, wherein the measuring probe has at least two cameras,further comprising: causing the optical properties of each camera to bedetermined by bundle adjustment.
 15. A method for determining theposition of control points in a network which is part of a system usedfor determining spatial coordinates of points, the system including: afixed array of a network of control points, a manually or mechanicallymanipulatable measuring probe containing at least one camera, themeasuring probe being designed for physical contact with a measuringpoint on an object which is to be measured via a fixedly mounted contactunit projecting from the measuring probe, which has a known, definedposition in the coordinate system of the measuring probe; and a systemunit for computing the position of the contact unit relative to acoordinate system defined by said control points on the basis of themeasuring probe's image making of the control points, and the knownposition of the contact unit in the coordinate system of the measuringprobe, wherein the distance between at least two of the control pointsis known, the method comprising: moving the camera or cameras of acalibrated measuring probe to at least two different directions ofobservation; recording in the camera or cameras an image of the fixedarray of the control points in the network from said differentdirections of observation; and establishing a table of coordinate valuesfor the control points in a coordinate system for the network defined bya selection of the control points.
 16. A method as disclosed in claim15, further comprising: causing the spatial coordinates of the controlpoints to be determined by bundle adjustment.
 17. A method forcalibrating a measuring probe in a system used for determining spatialcoordinates of points, the system including: a fixed array of a networkof control points, a manually or mechanically manipulatable measuringprobe containing at least one camera, the measuring probe being designedfor physical contact with a measuring point on an object which is to bemeasured via a fixedly mounted contact unit projecting from themeasuring probe, which has a known, defined position in the coordinatesystem of the measuring probe; and a system unit for computing theposition of the contact unit relative to a coordinate system defined bysaid control points on the basis of the measuring probe's image makingof the control points, and the known position of the contact unit in thecoordinate system of the measuring probe, where the position of controlpoints in a network which form part of the system are determinedconcurrently, the distance between at least two of the control pointsbeing known, the method comprising: moving the camera or cameras of thecalibrated measuring probe to at least two different directions ofobservation; recording in the camera or cameras an image of the fixedarray of the control points in the network from said differentdirections of observation; and establishing a table of coordinate valuesfor the control points in a coordinate system for the network defined bya selection of the control points and in the same computationcorrelating the images for establishing a number of correctionparameters or a calibration table corresponding to the relation betweenthe real and ideal image of a point.
 18. A method for calibrating thecontact unit on a measuring probe in a system used for determiningspatial coordinates of points, the system including: a fixed array of anetwork of control points, a manually or mechanically manipulatablemeasuring probe containing at least one camera, the measuring probebeing designed for physical contact with a measuring point on an objectwhich is to be measured via a fixedly mounted contact unit projectingfrom the measuring probe, which has a known, defined position in thecoordinate system of the measuring probe; and a system unit forcomputing the position of the contact unit relative to a coordinatesystem defined by said control points on the basis of the measuringprobe's image making of the control points, and the known position ofthe contact unit in the coordinate system of the measuring probe,comprising: moving the camera or cameras of the measuring probe to anumber of different observation directions or orientations, whilst itscontact unit remains in contact with a fixed reference point ormeasuring point on an object; registering the position and orientationof the measuring probe relative to a network of control points in thatin the camera or cameras there is recorded an image of the fixed arrayof the control points in the network for each of said directions ororientations; and combining registered position and orientation of themeasuring probe for each registration with information that the point ofcontact for the contact unit is the same, whereby the position of thecentre of contact unit is determined in relation to the network'scoordinate system and the internal coordinate system of the measuringprobe.
 19. A system as in claim 12 wherein the light source is a flashunit.